The effective and safe use of a knife requires occasional sharpening to maintain a keen edge. For a knife which is used at a single location, a sharpening device, such as a whetstone or a sharpening steel can be conveniently stored nearby. For a knife which is carried about, the carrying along of a separate sharpening device can be inconvenient, since the device can be bulky if it is a stone or can be easily lost if it is a smaller item, such as a tungsten carbide or ceramic device. For this reason, some knives designed for carrying have a sharpening device stored in or built into a sheath. The sharpening surface can be a feature of the sheath because the sheath can be moved relative to the knife. This is not the case for folding knives, since they typically need no sheath for carrying. Thus there is a need for a sharpening device which can be attached to a folding knife, but which is removable.